Predicted and observed impacts of COVID-19 lockdowns: two Health Impact Assessments in Scotland and Wales

Abstract Health Impact Assessment is a key approach used internationally to identify positive or negative impacts of policies, plans and proposals on health and well-being. In 2020, HIAs were undertaken in Scotland and Wales to identify the potential health and well-being impacts of the ‘stay at home’ and physical distancing measures implemented at the start of the coronavirus disease (COVID-19) pandemic. There is sparse evidence evaluating whether the impacts predicted in HIAs occur following policy implementation. This paper evaluates the impacts anticipated in the COVID-19 HIAs against actual observed trends. The processes undertaken were compared and predicted impacts were tabulated by population groups and main determinants of health. Routine data and literature evidence were collated to compare predicted and observed impacts. Nearly all health impacts anticipated in both HIAs have occurred in the direction predicted. There have been significant adverse impacts through multiple direct and indirect pathways including loss of income, social isolation, disruption to education and services, and psychosocial effects. This research demonstrates the value of prediction in impact assessment and fills a gap in the literature by comparing the predicted impacts identified within the HIAs with observed trends. Post-COVID-19 recovery should centre health and well-being within future policies and decisions. Processes like HIA can support this as part of a ‘health in all policies’ approach to improve the health and well-being of populations.


INTRODUCTION
Health Impact Assessment (HIA) is a key approach to promote 'Health in all policies' by assessing determinants of health and health equity likely to be affected by proposed policies, plans and proposals in all sectors. This should enable governments to ensure their policies promote health and health equity (Rogerson et al., 2020). HIA aims to identify potential impacts-positive or negative, intended and unintended-their scale and nature, and affected populations. It uses evidence and follows the five steps (Green et al., 2020;WHO, 2022) depicted in Box 1. HIA can support collaboration with stakeholders who provide insights and ensure ownership of action. (Leppo et al., 2013;WHO, 2014;Rogerson et al., 2020;Green et al., 2021).
HIAs and other impact assessments are best used to inform policies before decisions are taken and involve anticipating or predicting their potential impacts (Haigh et al., 2013). While prospective assessment is needed to inform decision-making (Davenport et al., 2006), the lack of direct evidence can lead some to question the validity of findings (Parry and Stevens, 2001). Retrospective validation of prediction in impact assessment is little addressed in the literature and discussion has focussed on the use of predictive tools (Hecky et al., 1984;George, 2000;Gontier et al., 2006;Molefe, 2017) or the challenges of evaluating and predicting impacts (Ali et al., 2009). For prediction in HIA specifically, the literature is even sparser (Petticrew et al., 2007;Veerman et al., 2007).
Evaluations of HIA can consider their accuracy in predicting the health and well-being impacts; how well stakeholders were engaged or their effectiveness in informing decisions (Parry and Kemm, 2005). Several studies have evaluated the effectiveness of HIAs in influencing decisions (Wismar et al. 2007;Dannenberg et al., 2008;Haigh et al., 2013Haigh et al., , 2015Nour et al., 2016;Bias and Abildso, 2017;Buregeya et al., 2020). Some studies have evaluated the HIA process including the evidence used (Tyler et al., 2019), consideration of equity (Povall et al., 2014;Buregeya et al., 2019) or level of participation (Thondoo et al., 2019). But we found none that assessed the accuracy of prediction by comparing the impacts predicted in an HIA with those observed after a policy or plan has been implemented.
The coronavirus disease  pandemic and the measures taken to control transmission, including requirements to stay at home, work from home, and socially distance, have had far-reaching effects on populations and society since March 2020 (Gautam and Hens, 2020;Public Health England, 2020;WHO, 2020;World Economic Forum, 2020;Chiesa et al., 2021;United Nations, 2021). These have affected health and inequalities beyond the direct morbidity and mortality caused by illness from the virus (Dyakova et al., 2021).
They have affected wider determinants of health, including the economy, the environment, social interaction, mental well-being and access to services such as education and health (Dahlgren and Whitehead, 2021). They have also had differential effects on population groups including older people, children, young people, women and those on low incomes (UK United Nations, 2020).
In 2020, during the first wave of COVID-19 in the UK, both the Scottish Health and Inequalities Impact Assessment Network (SHIIAN) and the Wales Health Impact Assessment Support Unit (WHIASU) carried out HIAs to identify the potential impacts of 'stay at home' and physical distancing measures ('lockdown') that were then being implemented (Douglas et al., 2020;Green et al., 2021a). Table 1 presents the policy measures that were assessed.
Both HIAs considered potential impacts on population health, well-being and health inequalities and made recommendations to mitigate negative impacts and enhance positive impacts. This paper aims to compare the predicted impacts identified in the two HIAs, with observed trends in both countries over the 18-month period after the start of the pandemic, between March 2020 and December 2021. The paper briefly compares the two HIAs, reports on a comparison of predicted and observed impacts, their accuracy, discusses reasons for any differences, and suggests implications for policy and practice.

METHODS
The methods and findings of both the Welsh and the Scottish HIAs have been reported previously (Douglas et al., 2020(Douglas et al., , 2020aGreen et al., 2020aGreen et al., , 2021a. Both assessed the 'stay at home' and physical or social distancing policies of the Welsh and Scottish Governments, and implemented in response to the COVID-19 pandemic to control the transmission of the virus. They followed the standard HIA process (Douglas, 2011;Bhatia et al., 2014;Winkler et al., 2020;WHO, 2022) and were undertaken in the first wave of the pandemic (March-May 2020). Table 2 summarizes the processes followed in the two HIAs. The Scottish HIA was completed in 1 week between 13 and 20 March 2020, before lockdown measures were initiated on 26 March. The HIA used a health impact checklist (Douglas, 2019) to identify potential mechanisms through which positive and negative impacts might arise. The five authors then collated routine data, systematic reviews, and other peer-reviewed and grey literature. They estimated for each impact an approximate number of people likely 3 to be affected, and a qualitative description of severity and direction of the impacts (positive or negative). There was no stakeholder engagement. The report presented the main areas of impact and recommended mitigation measures (Douglas et al., 2020a) and included a more detailed impacts table. The HIA was used as an initial framework and subsequently developed further as part of the Public Health Scotland COVID-19 Response.

Overview of the Scottish and Welsh HIAs
The Welsh HIA was completed in 2 months between April and 11 May 2020, from the start of lockdown and during the first few weeks of the first wave. The HIA used health and well-being and population group checklists (WHIASU, 2020) to identify mechanisms through which impacts might arise. The six authors collated routine data, systematic reviews, and peer-reviewed and grey literature. Stakeholder engagement involved interviews with 15 key stakeholders from Wales. The HIA team estimated for each impact the direction (positive/negative), likelihood, intensity/ severity and duration using previously validated definitions (Green et al., 2020). Reporting included an Executive Summary; a Main Report with an analysis of main areas of impact, recommended mitigation measures, and detailed impacts table; and a Supporting Information technical report (Green et al., 2020a). The HIA was used as an initial framework with an indication of the scale of some impacts and subsequently developed further as part of the Public Health Wales and Welsh  Response. This paper uses the areas of impact highlighted in the original HIA report.
Overall, the two HIAs followed similar processes and assessed broadly similar policies. However, the Welsh HIA gathered evidence from stakeholders and included a more comprehensive literature review and evidence-gathering process. The Scottish HIA was carried out immediately before the lockdown, whilst the Welsh HIA was conducted during the first 8 weeks. It also included a review and reflection on the process and recommended monitoring of future impacts.

Comparing HIA predictions with observed impacts
Firstly, the affected populations and main determinants or areas of impact identified in the two HIA reports were tabulated to compare the impacts identified between each HIA. Robust routine monitoring and survey data on relevant trends were then collated for each identified area of impact. This included Scottish and Welsh Government data, data from StatsWales, the Office for National Statistics, Public Health Wales reports and surveys such as the PHW Public Engagement Survey on Coronavirus Measures (Public Health Wales, 2022), Public Health Scotland reports

Scotland
The Welsh HIA was completed rapidly from 1 April to 11 May 2020, as the regulations were coming into force. These included the following: •Welsh Ministers, registered public health officials and police constables had the right to detain people infected or contaminated with coronavirus •required some business premises to close, and required those that were allowed to stay open, such as supermarkets, to put specific measures in place to ensure adequate social distancing •restricted the movements of individuals so that they were not allowed to leave the place they were living without a 'reasonable excuse'. The regulations included examples of these such as shopping for food, exercising once a day, getting medical help and travelling to work where it was not reasonable and practicable to work from home •closed places of worship, apart from in limited circumstances such as in relation to funerals •required Natural Resources Wales, local authorities, National Park Authorities and the National Trust to close public footpaths and access land, where the use of a path or land posed a high risk of spreading coronavirus.
The Scottish HIA was completed rapidly between 13 and 20 March 2020, before restrictions were approved or implemented. The authors assumed that measures to contain the virus would include all of some of the following: •Advising the whole population to self-isolate at home if they or their family are symptomatic •Bans on social gatherings •Stopping flights and public transport •Closure of 'non-essential' workplaces (i.e. beyond the health and social care sector, utilities and the food chain) with continued working from home for those that can •Closure of schools, colleges and universities •Prohibition of all 'non-essential' population movement •Limiting contact for special populations (i.e. care homes, prisons) 4 L. Green et al. and surveys. The authors also drew on other surveys in Wales Scotland-and UK-wide. These sources were used to compare the direction of change for each determinant with the direction (positive or negative) predicted in the HIAs. We did not attempt to assess whether the judgments of severity were accurate as these were mostly qualitative descriptors. We graded the strength of evidence available for each impact as High (rated as 1) for government or public health institute data and for peer-reviewed papers and Low for other sources (rated as 2).

Comparison of predicted and observed impacts
Tables 3 and 4 present a summary of the predicted impacts and whether they were observed and accurate, indicating the strength of supporting evidence.
Supplementary Tables S1 and S2 provide further detail. For most of the impacts, high-grade evidence was available from government or public health institute sources supporting the prediction. For both HIAs, most predicted impacts were observed in the expected direction (positive or negative). As predicted, there were negative impacts on income, particularly for people already on low incomes, which were only partially mitigated by the Coronavirus Job Retention scheme and other measures (Widnall et al., 2020;ONS, 2021). Social isolation led to high levels of loneliness (Groarke et al., 2020;Public Health Wales, 2020;WCPP, 2021; and there is evidence of increased child abuse and domestic abuse (Davenport et al., 2020;ONS, 2020;Scottish Government, 2020, as predicted in both HIAs. The HIAs predicted that disruption and unwillingness to attend health settings would affect the care of non-COVID-19 conditions, and this has occurred (Scottish Public Health Observatory, 2021; Welsh Parliament, 2021a). Similarly, the HIAs predicted disruption to education would increase educational inequalities and there is evidence that this has happened (Public Health Wales, 2021; Scottish Government, 2021). The predicted short-term fall in car and other journeys occurred, resulting in improved air quality and other environmental impacts (Transport Scotland, 2021;Welsh Parliament, 2021b). But, as also predicted, reluctance to use public transportation has led to higher car travel and this now accounts for a higher proportion of journeys. Both HIAs predicted negative effects on mental well-being through high levels of anxiety (Public Health Wales, 2020; Scottish Government, 2020a;Mind Cymru, 2021). They also predicted positive effects on community cohesion through collective individual and neighbourhood responses and there is evidence of high levels of community support during the pandemic (Edinburgh Community Health Forum, 2020; Public Health Wales, 2020). The HIAs predicted potential negative impacts on ethnic minority populations and there has been an observed increase in hate crime and reports of harassment affecting people of Asian ancestry and disabled people (BBC, 2021; UK Government, 2021). The Welsh HIA predicted exacerbation of these impacts by crowded living conditions, and there is evidence of an increase in household disputes (Woodfine et al., 2021). The Scottish HIA identified the negative impact of restricted access to greenspace, and surveys found that use of greenspace reduced during the 2020 lockdown period although respondents reported it benefited their mental health (Public Health Scotland, 2021a). The Scottish HIA identified the potential for unrest, whereas the Welsh HIA predicted a reduction in overall crime and increased trust in the police. The Welsh HIA was more correct, as most forms of crime reduced and there has been no significant unrest in either nation, although there were increases in some types of crime such as domestic abuse, as noted above, and virtual fraud (Public Health Wales, 2020; Scottish Government, , 2021a. The HIAs also differed in their assessment of likely impacts on health-related behaviour, and the observed impacts have also been mixed (Public Health Scotland, 2021Public Health Scotland, 2021Public Health Wales, 2020). Both HIAs identified the potential for increases in health-harming behaviours, for example snacking and alcohol use. The Scottish HIA predicted a reduction in physical activity whereas the Welsh HIA predicted that increased appreciation of physical activity may lead some to increase physical activity. The available data suggest that some people adopted healthier behaviours, but a higher proportion adopted less healthy behaviours, including physical activity for which larger declines were seen in younger, ethnic minority and unemployed populations (Obesity Action Scotland, 2020;Public Health Wales, 2020). The Welsh HIA also identified a reduction in street sleeping (Woodfine et al., 2021) and increased digital connectivity (Public Health Wales, 2020), both of which occurred, partly due to mitigation measures implemented early in the pandemic.
Both HIAs identified population groups that would be particularly vulnerable to negative impacts. Most of these populations have been disproportionately affected by the anticipated impacts. The only exception is that both HIAs predicted that older people would bear a higher impact on mental health than other groups. Despite bearing a disproportionately much higher burden of direct morbidity and mortality from COVID-19, older people have reported lower levels of anxiety, loneliness and hopelessness than younger age groups (Office for Health Improvement and Disparities, 2021).

DISCUSSION
This paper has shown that most of the health impacts anticipated in the Scottish and Welsh HIAs have occurred in the predicted direction, using monitoring data up to December 2021. As predicted, there have been significant adverse impacts through multiple pathways including loss of income and employment; mental health and well-being impacts of social isolation, stress and anxiety; family stress and increased violence against women and domestic abuse; disruption to health and other services; educational disruption; and a reluctance to use public transport. These have disproportionately affected population groups who were already disadvantaged, for example, women, older people, those on low incomes, children and young people. The impacts on health-related behaviours have been more mixed, for example some people increased physical activity or alcohol intake, and some reduced it. The main positive impact identified in the Scottish HIA was the potential for an increased sense of community, which has been demonstrated. The Welsh HIA identified further positive impacts reflecting mitigation measures early in the pandemic, such as provision of accommodation for homeless people and the increased use of digital technology. We are not aware of any other evaluations of HIAs that assessed whether the anticipated impacts proved to be correct after the proposed policy was implemented. Assessing the predictive accuracy of an HIA is difficult, as the impacts that occur after implementation may be altered by responses taken after the HIA has been completed (Parry and Stevens, 2001;Petticrew et al., 2007). This includes measures taken to implement the HIA recommendations, so ironically if an HIA is effective in influencing policy this makes it more difficult to evaluate the accuracy of its predictions. In this case, several interventions were implemented to help mitigate wider impacts, although not directly as a result of these HIAs. These include the Coronavirus Job Retention Scheme ('furlough') providing economic support to employers and employees (House of Commons Library, 2021a, 2021b), a temporary increase in universal credit (Department for Work and Pensions, 2021) and providing free laptops to some children and young people to support online learning (Department for Education, 2020). These measures will have mitigated some of the anticipated negative impacts, but significant residual impacts still clearly occurred, for example those on low incomes bore strongly negative impacts despite employment support (Mental Health Foundation, 2020).
Both HIAs followed the standard HIA process but differed in timing and depth. The earlier timing of the Scottish HIA potentially gave more opportunity to inform early mitigation measures, whereas the later Welsh HIA allowed early evidence of emerging impacts to suggest changes to mitigate or maximize these. The Scottish HIA was completed very rapidly in 1 week and included relevant data and literature evidence but no stakeholder involvement. The more comprehensive Welsh HIA allowed a fuller characterization of the likelihood, significance and duration of each impact, using criteria validated in previous HIAs (Green et al., 2020).
Finally, the HIAs identified very similar impacts with some differences (Tables 3 and 4; Supplementary  Tables S1 and S2). The Welsh HIA identified more opportunities for positive health and well-being. This included the potential for home working to promote better work/life balance and flexible working; increased family connectivity; and the potential to develop a more sustainable economic model prioritizing health and well-being. The main area in which they reached opposing conclusions was the impact on crime. The Welsh HIA predicted a crime reduction, based on police reports during the early part of the pandemic. Conversely, the Scottish HIA predicted a potential increase in crime and social disorder due to discontent about the pandemic response and reduced policing capacity. In practice, there was little significant disorder in the UK (although events elsewhere show the potential for this to arise). Crime overall fell during restrictions, especially violent crimes associated with the night-time economy, but some crimes such as domestic abuse, child abuse and virtual fraud increased (Scottish Government, 2020; Office for National Statistics, 2022). A more detailed HIA in Scotland may have predicted these nuances. Both HIAs identified the potential for an increase in health-harming behaviours, but they diverged in their prediction of impacts on physical activity. Whereas the Scottish HIA identified the potential for reduced physical activity because of closed sports facilities and less utilitarian active travel, the Welsh HIA identified increased appreciation of the importance of physical activity as population movements outside of the home were restricted (PHW, 2020). In both countries and across the UK, the impacts on health-related behaviours have been mixed, with polarization between populations and a likely increase in inequalities (Convention of Scottish Local Authorities and Scottish Public Health England, 2020a). Some negative consequences were not foreseen by either HIA, i.e. the Welsh HIA identified positive opportunities for local tourism but not the accompanying negative impacts of increased tourist traffic, increased accommodation prices and environmental damage (Schofields Insurance, 2020;Christian, 2021;Department for Transport, 2021).
The strengths of this evaluation are that it is original in evaluating the accuracy of predictions in the HIAs, and the broad range of data sources used to assess whether anticipated impacts emerged in the direction expected. There are limitations to the available data for some impacts, particularly health-related behaviours and differences between population groups. Also, some impacts may emerge later-for example, longerterm impacts on unemployment, predicted by both HIAs, are not yet clear. The HIAs presented a high-level characterization of different types of impacts and the evaluation has not sought to determine whether judgments about scale or severity of impact were correct. The authors were involved in the HIAs, which might bias our conclusions. However, the two HIAs were completed independently and this evaluation, involving authors of both HIAs, encouraged cross-scrutiny and appraisal and used robust sources where available. As noted above, anticipated impacts will have been affected by mitigation measures and further work is needed to explore the extent to which the HIAs were able to influence policy responses.

Implications for HIA practice
This evaluation demonstrates that even a very rapid HIA can correctly predict many relevant impacts on health and equity before a policy is implemented. The breadth of the impacts and their differential effects reinforce the need for a holistic approach, enabling HIAs to identify potential impacts affecting different populations through multiple pathways. Comparison between the HIAs demonstrates the benefit of stakeholder involvement and a more detailed review of supporting evidence. These are routinely advocated to enable robust analysis (Mindell et al., 2004(Mindell et al., , 2010(Mindell et al., , 2014Tamburrini et al., 2011;Negev et al., 2013). Evidence from stakeholders, who were being affected by impacts that had not yet been captured in literature or statistics, provided more certainty and depth. In every HIA, there is a need to balance the available resources and capacity, the level of detail needed to provide robust conclusions and the need for timeliness of recommendations to influence decision-making and inform actions.
Both HIAs were used as a framework for initial mitigation and were then developed further. This allowed the initial findings to be used while emerging impacts were explored. This shows that HIAs can support an ongoing, collaborative 'Health in All Policies' approach working across sectors rather than being just a one-off assessment (Rogerson et al., 2020;Green et al., 2021).
This evaluation shows the potential to monitor observed impacts following an HIA, using routine data. Although most HIA guidance suggests that HIA should include monitoring and evaluation (Quigley and Taylor, 2004;Harris et al., 2007;Dannenberg et al., 2008; National Research Council (US) Committee on Health Impact Assessment, 2011;Pyper et al., 2021), in practice this is often not done. Undoubtedly, many of the impacts anticipated in these HIAs were partially mitigated by other measures implemented alongside social distancing restrictions. However, this paper shows that it is still possible to assess whether the HIA predictions were broadly correct. Further similar evaluations could help demonstrate whether, and when, HIAs are effective in predicting future impacts and improve future practice. This paper assessed national-level HIAs for which national data were readily available, but relevant data are also often disaggregated to more local levels thus making this approach replicable locally. HIAs should highlight the priority indicators to monitor, so they can be collated following proposal implementation.
The close match between predicted and observed negative impacts raises the question of whether the HIAs were effective in informing action to mitigate these. Were the HIAs ignored, or did they inform actions that in practice only partially reduced the impacts? The legal context is important as the ability to more fully mitigate some impacts may have been beyond the powers of the devolved Welsh and Scottish governments. A consideration for future HIAs is to prioritize the impacts that are most amenable to action and specify the authorities with power to implement these. Further research could explore how HIAs are used by policy-makers to enhance their effectiveness in influencing action.
In both HIAs, some or all authors worked in the national public health organization. This enabled access to relevant evidence, for example, health observatory data and statistics, but also more importantly allowed the findings to be used in the relevant organizations' responses to the wider impacts of the pandemic. Both HIAs and findings were also published in academic journals, reaching a wider audience (Douglas et al., 2020;Green et al., 2021a).

Implications for policy
HIAs are themselves an intervention that aim to influence the outcome of the policies assessed (Mindell et al., 2014). The finding that even very rapid HIAs, such as the Scottish example, can effectively predict a wide range of impacts on health also supports the more widespread use of HIA in policymaking. Routine use of HIA could identify unanticipated potential health impacts before, or as, policies are implemented, offering the opportunity to mitigate adversely and enhance positive impacts in 'real time' (Green et al., 2021a). Evaluations showing evidence of validated prediction should increase commissioners' confidence to use HIA.
The data also highlight the range of negative impacts of the pandemic and their differential effects. There is a clear need for continuing action to address these residual health impacts in the post-pandemic period. Processes such as HIA can help to ensure actions are well designed to enhance their positive effects, avoid unanticipated harms and are targeted to the populations most affected.

CONCLUSION
This paper evaluated the impacts identified in two HIAs that assessed the impact of 'lockdown' in Scotland and Wales in 2020. It demonstrates the value of prediction in HIA and fills a gap in the literature by comparing predicted with observed impacts. The rapid Scottish and more comprehensive Welsh approaches both have value, with the stakeholder involvement and more comprehensive evidence review allowing more detailed characterization of the impacts to inform decisions and action.
The pandemic has raised the profile of public health more widely. The use of processes such as HIA can build on this and inform decisions based on evidence and predictive analysis. Evaluations like this could increase confidence in prospective HIA. Post-COVID-19 recovery and renewal should allow health and well-being should be centred within future policies and decisions. Processes such as HIA can support this and form a key part of a 'health in all policies' approach .

SUPPLEMENTARY MATERIAL
Supplementary material is available at Health Promotion International online.
Box 1: The HIA process Step 1: Screening to determine whether to complete an HIA. This includes consideration of whether there are likely to be effects on health and whether there is scope for changes to improve these impacts.
Step 2: Scoping the boundaries of the assessment, including timeframes, resources, key stakeholders to engage with, evidence collection methods and key determinants and populations of focus.
Step 3: Appraisal of evidence, which is triangulated and analysed. This evidence can include peer-reviewed and grey literature, stakeholder evidence and routinely gathered statistics and data for example, government statistics and reporting.
Step 4: Recommendations and reporting to inform decision makers, including the construction of a report which includes the findings and any recommended actions that should be taken to maximize the positive impact and mitigate any negative impact.
Step 5: Review and reflection including monitoring and evaluation. This involves highlighting milestones to measure any changes in impact or if the predicted impacts were observed, reviewing the process and any impact which it may have had on decisions and future policies.

FUNDING
This work did not receive any specific funding from any funding agency in the public, commercial or notfor-profit sectors. For the purpose of open access, the author has applied a Creative Commons Attribution (CC BY) licence to any author accepted manuscript version arising from this submission.

ETHICAL APPROVAL
This work used only published aggregated data and evidence, and did not require ethical approval.